Self-Organizing Teams in Online Work Settings

As the volume and complexity of distributed online work increases, the collaboration among people who have never worked together in the past is becoming increasingly necessary. Recent research has proposed algorithms to maximize the performance of such teams by grouping workers according to a set of predefined decision criteria. This approach micro-manages workers, who have no say in the team formation process. Depriving users of control over who they will work with stifles creativity, causes psychological discomfort and results in less-than-optimal collaboration results. In this work, we propose an alternative model, called Self-Organizing Teams (SOTs), which relies on the crowd of online workers itself to organize into effective teams. Supported but not guided by an algorithm, SOTs are a new human-centered computational structure, which enables participants to control, correct and guide the output of their collaboration as a collective. Experimental results, comparing SOTs to two benchmarks that do not offer user agency over the collaboration, reveal that participants in the SOTs condition produce results of higher quality and report higher teamwork satisfaction. We also find that, similarly to machine learning-based self-organization, human SOTs exhibit emergent collective properties, including the presence of an objective function and the tendency to form more distinct clusters of compatible teammates

Listener Modeling and Context-aware Music Recommendation Based on Country Archetypes

Music preferences are strongly shaped by the cultural and socio-economic background of the listener, which is reflected, to a considerable extent, in country-specific music listening profiles. Previous work has already identified several country-specific differences in the popularity distribution of music artists listened to. In particular, what constitutes the "music mainstream" strongly varies between countries. To complement and extend these results, the article at hand delivers the following major contributions: First, using state-of-the-art unsupervised learning techniques, we identify and thoroughly investigate (1) country profiles of music preferences on the fine-grained level of music tracks (in contrast to earlier work that relied on music preferences on the artist level) and (2) country archetypes that subsume countries sharing similar patterns of listening preferences. Second, we formulate four user models that leverage the user's country information on music preferences. Among others, we propose a user modeling approach to describe a music listener as a vector of similarities over the identified country clusters or archetypes. Third, we propose a context-aware music recommendation system that leverages implicit user feedback, where context is defined via the four user models. More precisely, it is a multi-layer generative model based on a variational autoencoder, in which contextual features can influence recommendations through a gating mechanism. Fourth, we thoroughly evaluate the proposed recommendation system and user models on a real-world corpus of more than one billion listening records of users around the world (out of which we use 369 million in our experiments) and show its merits vis-a-vis state-of-the-art algorithms that do not exploit this type of context information.Comment: 30 pages, 3 tables, 12 figure

An Examination of the Work Practices of Crowdfarms

Crowdsourcing is a new value creation business model. Annual revenue of the Chinese market alone is hundreds of millions of dollars, yet few studies have focused on the practices of the Chinese crowdsourcing workforce, and those that do mainly focus on solo crowdworkers. We have extended our study of solo crowdworker practices to include crowdfarms, a relatively new entry to the gig economy: small companies that carry out crowdwork as a key part of their business. We report here on interviews of people who work in 53 crowdfarms. We describe how crowdfarms procure jobs, carry out macrotasks and microtasks, manage their reputation, and employ different management practices to motivate crowdworkers and customers

On how technology-powered storytelling can contribute to cultural heritage sustainability across multiple venues-Evidence from the crosscult H2020 project

Sustainability in Cultural Heritage (CH) is a complex question that needs to be addressed by a group of experts tackling the different issues. In this light, the present work wishes to provide a multi-level analysis of the sustainability in CH, using as an example a recent European H2020 project (CrossCult) and the lessons learnt from its design, implementation and evaluation. The sustainability of CH has qualitatively changed over the last few years, under the developments in digital technology that seems to affect the very nature of the cultural experience. We discuss sustainability in venues using digital technologies, covering a span of needs of small/unknown and large/popular venues, which try to enhance the visitor experience, attract visitors, form venue networks, etc. Moreover, we explore issues of sustainability of digital content and its re usability through holistic design. Aspects of technology, human networks and data sustainability are also presented, and we conclude with the arguments concerning the sustainability of visitor reflection, the interpretation of social and historical phenomena and the creation of meaning.This research was funded by the European Union’s Horizon 2020 research and innovation programme under grant agreement No 693150. The authors from the University of Vigo got further support from the European Regional Development Fund (ERDF) and the Galician Regional Government under agreement for funding the AtlantTIC Research Center for Information and Communication Technologies, as well as the Ministerio de Educación y Ciencia (Gobierno de España) research project TIN2017-87604-R

Lessons Learned About Designing and Conducting Studies From HRI Experts.

The field of human-robot interaction (HRI) research is multidisciplinary and requires researchers to understand diverse fields including computer science, engineering, informatics, philosophy, psychology, and more disciplines. However, it is hard to be an expert in everything. To help HRI researchers develop methodological skills, especially in areas that are relatively new to them, we conducted a virtual workshop, Workshop Your Study Design (WYSD), at the 2021 International Conference on HRI. In this workshop, we grouped participants with mentors, who are experts in areas like real-world studies, empirical lab studies, questionnaire design, interview, participatory design, and statistics. During and after the workshop, participants discussed their proposed study methods, obtained feedback, and improved their work accordingly. In this paper, we present 1) Workshop attendees' feedback about the workshop and 2) Lessons that the participants learned during their discussions with mentors. Participants' responses about the workshop were positive, and future scholars who wish to run such a workshop can consider implementing their suggestions. The main contribution of this paper is the lessons learned section, where the workshop participants contributed to forming this section based on what participants discovered during the workshop. We organize lessons learned into themes of 1) Improving study design for HRI, 2) How to work with participants - especially children -, 3) Making the most of the study and robot's limitations, and 4) How to collaborate well across fields as they were the areas of the papers submitted to the workshop. These themes include practical tips and guidelines to assist researchers to learn about fields of HRI research with which they have limited experience. We include specific examples, and researchers can adapt the tips and guidelines to their own areas to avoid some common mistakes and pitfalls in their research

Achieving Self-Directed Integrated Cancer Aftercare (ASICA) in melanoma: protocol for a randomised patient-focused pilot trial of delivering the ASICA intervention as a means to earlier detection of recurrent and second primary melanoma.

BACKGROUND: Melanoma is common; 15,906 people in the UK were diagnosed with melanoma in 2015 and incidence has increased fivefold in 30 years. Melanoma affects old and young people, with poor prognosis once metastatic. UK guidelines recommend people treated for cutaneous melanoma receive extended outpatient, hospital follow up to detect recurrence or new primaries. Such follow up of the growing population of melanoma survivors is burdensome for both individuals and health services. Follow up is important since approximately 20% of patients with early-stage melanoma experience a recurrence and 4-8% develop a new primary; the risk of either is highest in the first 5 years. Achieving Self-directed Integrated Cancer Aftercare (ASICA) is a digital intervention to increase total-skin-self-examination (TSSE) by people treated for melanoma, with usual follow up. METHODS: We aim to recruit 240 adults with a previous first-stage 0-2C primary cutaneous melanoma, from secondary care in North-East Scotland and the East of England. Participants will be randomised to receive the ASICA intervention (a tablet-based digital intervention to prompt and support TSSE) or control group (treatment as usual). Patient-reported and clinical data will be collected at baseline, including the modified Melanoma Worry Scale (MWS), the Hospital Anxiety and Depression Scale (HADs), the EuroQoL 5-dimension 5-level questionnaire (EQ-5D-5 L), and questions about TSSE practice, intentions, self-efficacy and planning. Participants will be followed up by postal questionnaire at 3, 6 and 12 months following randomization, along with a 12-month review of clinical data. The primary timepoint for outcome analyses will be12 months after randomisation. DISCUSSION: If the ASICA intervention improves the practice of TSSE in those affected by melanoma, this may lead to improved psychological well-being and earlier detection of recurrent and new primary melanoma. This could impact both patients and National Health Service (NHS) resources. This study will determine if a full-scale randomised controlled trial can be undertaken in the UK NHS to provide the high-quality evidence needed to determine the effectiveness of the intervention. ASICA is a pilot study evaluating the effectiveness of the practice of digitally supported TSSE in those affected by melanoma. TRIAL REGISTRATION: Clinical Trials.gov, NCT03328247 . Registered on 1 November 2017

Improving Fairness and Transparency for Artists in Music Recommender Systems

Streaming services have become one of today's main sources of music consumption, with music recommender systems (MRS) as important components. The MRS' choices strongly influence what users consume, and vice versa. Therefore, there is a growing interest in ensuring the fairness of these choices for all stakeholders involved. Firstly, for users, unfairness might result in some users receiving lower-quality recommendations in terms of accuracy and coverage. Secondly, item provider (i.e. artist) unfairness might result in some artists receiving less exposure, and therefore less revenue. However, it is challenging to improve fairness without a decrease in, for instance, overall recommendation quality or user satisfaction. Additional complications arise when balancing possibly domain-specific objectives for multiple stakeholders at once. While fairness research exists from both the user and artist perspective in the music domain, there is a lack of research directly consulting artists - -with Ferraro et al. (2021) as an exception. When interacting with recommendation systems and evaluating their fairness, the many factors influencing recommendation system decisions can cause another difficulty: lack of transparency. Artists indicate they would appreciate more transparency in MRS - -both towards the user and themselves. While e.g. Millecamp et al. (2019) use explanations to increase transparency for MRS users, to the best of our knowledge, no research has addressed improving transparency for artists this way

Child's Bonding and Self-Disclosing with a Robot in Family Care

This extended research abstract for the Doctoral Consortium at IDC 2021 describes a 5-year PhD project, started November 2019, on self-disclosure in child-robot interaction in the field of child and family care. The research design embraces a bottom-up participatory design approach including all stakeholders, based on qualitative as well as quantitative methods. This PhD research is guided by Dr. M.M.A. de Graaf and Prof. dr. ir. J.F.M. Masthoff

Ratings in, rankings out. Keep it simple, they said. But we need more than that

Among the many viable research questions in the field of recom-mender systems, a frequently addressed problem is to accurately predict the relevance of individual items to users, with the goal of presenting the assumedly most relevant ones as recommendations. Typically, we have users' (explicit or implicit) ratings as input and rankings of items as output. Complex enough, yet too simplistic to reflect reality and indeed meet the various demands in practice. We have learned that "context matters". But what does it mean? What is the context that matters? And how do we get the relevant signals? It is more than what we currently ascribe to and reflect in what we call "context-aware recommender systems". Let's have a view to related fields that deal with context as deeply complex input. And on the output side, we have individual items and also item bundles, complementaries, sequences, repeated recommendations, etc. What do we actually want to present? And how? For who? And why? A ranked list as output may seem like an appropriate one-size-fits-all solution, does it? In this talk, I will reflect on the complexity of our research field, reach out to related fields such as context-aware computing and pervasive advertising for inspiration, and I will raise a lot of questions that have yet to be answered

Multi-Method Evaluation: Leveraging Multiple Methods to Answer What You Were Looking For

Research in the field of information retrieval and recommendation mostly focuses on one single evaluation method and one single quality objective. On the one hand, many research endeavors focus on system-centric evaluation from an algorithmic perspective and consider the context of use only to a minor extent. On the other hand, there are research endeavors focusing on user-centric approaches to the design and evaluation of systems. However, algorithmic quality and perceived quality of user experience do not necessarily match. Thus, it is essential for system evaluation to substantially integrate multiple evaluation methods that cover a variety of relevant aspects and perspectives. Only such an integrated combination of methods may lead to a deep understanding of users, their behavior, and experience in their interaction with a system. This half-day tutorial follows the objective to raise awareness in the CHIIR community concerning the significance of using multiple methods in the evaluation of information retrieval and recommender systems. The tutorial illustrates the "blind spots'' when using single methods. It introduces the concept of "multi-method evaluation'' and discusses its benefits and challenges. While multi-method evaluations may be designed very flexibly, the tutorial presents broadly-defined basic options of how multiple methods may be integrated in an evaluation design. In group work, participants are encouraged to select and fine-tune a specific design that best matches their research endeavor's purpose